feat(chat): hand run() a streamText with the managed options already applied - #4884
feat(chat): hand run() a streamText with the managed options already applied#4884ericallam wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: f36bb10 The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Spreading chat.toStreamTextOptions() is the integration point for six things:
the managed prompt and its cache control, the resolved model, the prompt's
sampling config, telemetry, the skill tools, and the prepareStep that delivers
steering, compaction and injected context. Forgetting the spread drops all six
in silence, and spread order decides whether passing your own tools or
prepareStep clobbers the managed ones.
run() now receives a streamText with those options applied, so the managed
state cannot be lost by omission and the merge happens inside rather than at
the call site: tools go into the helper so skills survive, a caller system
becomes the base the prompt and injections append to, and a caller prepareStep
composes after the managed one instead of replacing it.
The signature is borrowed with typeof import("ai").streamText rather than
restated, so it resolves to whichever of ai v5/v6/v7 the user installed. The
runtime value rides the existing ESM/CJS shim that already isolates value
imports from ai.
PROTOTYPE. Typechecks and passes the suite on ai@6.0.116 and ai@7.0.66, but
adds a public registry option, does not settle what happens when caller and
managed system are both structured, and has no test for the composed
prepareStep.
4984f70 to
f36bb10
Compare
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
Summary
Every
run()had to spreadchat.toStreamTextOptions(), and leaving it out dropped six things with no error: the managed prompt and its cache control, the registry-resolved model, the prompt's sampling config, telemetry, the skill tools, and theprepareStepthat delivers steering, compaction and injected context.Before:
After:
streamTextcomes fromrun's argument and shadows the one imported fromai, so the correct call is now the shorter one and the managed options cannot be lost by omission.chat.toStreamTextOptions()is unchanged and still supported, and is still the only option in a custom agent.What changes when your options collide with the managed ones
Spread order decides the outcome today, and losing is silent:
The managed
streamTextmerges instead.toolsare passed into the helper so skill tools survive, and aprepareStepyou pass runs after the managed one rather than replacing it. Everything else you name is left alone and wins, telemetry included.systemis the exception: it can be set onchat.agent({ system }), throughchat.prompt.set(), or at the call site, but only in one of them. Two at once throws and names the one that already owns it. No shape merges two system values across every supported AI SDK version, since v5 rejects an array of blocks and a structured block carries the provider options that make prompt caching work.onAction
A response produced from an action gets the same
streamText. Before, a regenerate answered with no system prompt and no skill tools, so the replacement answer came from a differently configured model than every other turn.Before:
After:
The only edit is the destructure. The two calls look the same and produce answers configured differently.
chat.headStart and chat.startHeadStart
buildStreamTextOptionssuppliesmessages,stopWhen: stepCountIs(1)andabortSignal. Step 1 belongs to the route handler and step 2 onward to the agent, so re-settingstopWhenafter a spread hands over a stream that has already run past step 1.Before:
After:
Passing
messages,stopWhenorabortSignalto thatstreamTextis a type error, with a runtime throw behind it for JavaScript callers. The old shape only warned in prose.Also in here
chat.agent()takessystem,registry,cacheControlandsystemProviderOptions, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site.ChatStreamTextis exported for typing a loop factored out ofrun.The signature is taken from the AI SDK's own declaration:
The peer range spans
aiv5, v6 and v7, whose options differ.typeofresolves to whichever version is installed, so generics and tool inference are the caller's own and a v8 option needs no change here.Verification
Typecheck and the full suite pass on both
ai@6.0.116andai@7.0.66. The option merge is a pure function so the merged object can be asserted directly, which is howexperimental_telemetrybeing dropped was caught: moststreamTextoptions never reach the provider, so a test that observes the model cannot see them.Run end to end against a deployed agent with every
runrewritten to the new form and no spread anywhere: steering, undo across a cold boot, and regenerate all still pass, a caller's ownprepareStepruns while managed steering still fires inside the turn, and consecutive injections arrive one per turn. The handover-owned options are pinned by@ts-expect-errorassertions in a typechecked test rather than only by the runtime throw.